Relativistic few-cycle mid-infrared (mid-IR) pulses are unique tools for strong-field physics and ultrafast science, but are difficult to generate with traditional nonlinear optical methods. Here, we propose a scheme to generate such pulses with high efficiency via plasma-based frequency modulation with a negatively chirped laser pulse (NCLP). The NCLP is rapidly compressed longitudinally due to dispersion and plasma etching, and its central frequency is downshifted via photon deceleration due to the enhanced laser intensity and plasma density modulations. Simulation results show that few-cycle mid-IR pulses with the maximum center wavelength of $7.9\;\unicode{x3bc} \mathrm{m}$ and pulse intensity of ${a}_{\mathrm{MIR}}=2.9$ can be generated under a proper chirp parameter. Further, the maximum energy conversion efficiency can approach 5.0%. Such a relativistic mid-IR source is promising for a wide range of applications.